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ABSTRACT 



Connectionist models apply various mathematical rules within 
neural network computer simulations in an effort, among other things, to 
mimic and describe human memory associations and learning. Learning involves 
the interaction of information provided by instruction with existing 
information already in the learner's memory (Ausubel , 1968; Bruner, 1990). 
When a learner commits to a lesson response, that response reflects the 
learner's immediate understanding of that instructional instance, thus 
initial lesson response (ILR) provides a measure of a learner's existing 
information. Describing what happens to memory traces of ILRs that are errors 
is necessary for determining whether errors interfere with attaining correct 
responses, and so is one key to understanding how feedback works. Among a 
number of connectionist learning rules, the delta rule (Shanks, 1995; Widrow 
& Hoff, 1960) is one of the simplest and most common that includes the 
effects of feedback on learning. Clariana (1999) has suggested that a 
connectionist approach using the delta rule can be used to predict posttest 
memory activation levels of ILR errors and of correct responses for immediate 
and delayed feedback. This experimental investigation provides empirical 
support for the potential of a connectionist approach to predict 
instructional feedback effects. Posttest data from high school students is 
compared to values predicted by the delta rule. Results and implications are 
provided. (Contains 27 references.) (AEF) 
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Abstract 

This experimental investigation provides empirical support for the potential of a connectionist approach to 
predict instructional feedback effects. Posttest data from high school students is compared to values predicted by 
the delta rule. Results and implications are provided. 

Learning involves the interaction of information provided by instruction with existing information already 
in the learner’s memory (Ausubel, 1968; Bruner, 1990). When a learner commits to a lesson response, that response 
reflects the learner’s immediate understanding of that instructional instance, thus initial lesson response (ILR) 
provides a measure of a learner’s existing information. 

Describing what happens to memory traces of ILRs that are errors is necessary for determining whether 
errors interfere with attaining correct responses, and so is one key to understanding how feedback works. Clariana 
(1999) has suggested that a connectionist approach using the delta rule can be used to predict posttest memory 
activation levels of ILR errors and of correct responses for immediate and delayed feedback. This investigation 
provides data that supports his approach. But first, what is a connectionist approach? 

Connectionist Description of Feedback Timing Effects 

Connectionist models apply various mathematical rules within neural network computer simulations in an 
effort, among other things, to mimic and describe human memory associations and learning. The theory includes 
several families of models, such as simple feedforward networks, pattern associators, multi-layer networks with 
backpropagation, competitive networks, and recurrent networks, that differ slightly in how the nodes of the network 
are interconnected, but which differ greatly in the kind of processing that they are able to do (see McLeod, Plunkett, 
& Rolls, 1998, for more detail). Neural networks have been shown to be capable of pattern matching, pattern 
completion, retrieval by content, recognition, prototype extraction, and classification to name a few (Haberlandt, 
1997). 

For example, Seidenberg and McClelland (1989) trained a computer neural network to read aloud all 
English monosyllabic words (about 3,000 words). After 250 training epochs, the model could correctly pronounce 
97% of the 3, 000 words in the training set. This neural network was able to accomplish this task without a local 
lexical store and more importantly without being given a set of rules. Elman (1993) trained a neural network with 
sentences rather than words, and was able to show that the network could satisfy long-distance grammatical 
dependencies (matching syntax). Plunkett and Marchman (1993; 1996) have modeled early lexical development 
which parallels that observed in children. Their neural network model that produces past tense forms of regular and 
irregular verbs has challenged the language acquisition orthodoxy that language learning depends on both innate 
pre-wiring of the system and on learning symbolic rules of the language. 

Among a number of connectionist learning rules, the delta rule (Shanks, 1995; Widrow & Hoff, 1960) is 
one of the simplest and most common that includes the effects of feedback on learning. The delta rule describes the 
change in association weight, termed Aw, between an input unit and an output unit at each learning trial, as: 

Awj 0 — oc a in (t 0 - a out ) 

where a is the learning rate parameter, a in is the activation level of input units, ^ is the desired response (the t refers 
to "teacher", in this case ^ is item feedback), and a^ is the activation level of the output units (Shanks, 1995). In 
instructional terms, learning is an increase in association, that is, an increase in Aw io between the stimulus (a in ) and 
the correct response (a^),’ with a relative decrease in association, that is, a decrease in Aw io for incorrect responses. 

To apply the delta rule in this study, following Clariana (1999), this investigation assumes that lesson 
average item difficulty values are reasonable estimates of the association weights of the correct responses. Item 
difficulty (p) is defined as the proportion (p g ) of individuals who answer an item (g) correctly (item difficulty 
notation convention from Crocker & Algina, 1986). For example an item difficulty of .20 indicates that 20% of the 
learners responded correctly to that item. Item difficulty values range from 0.00 to 1.00 with low values indicating 
difficult items and high values indicating easy items. Using lesson average item difficulty values as a group's 
estimate of the initial lesson item a out association weight seems reasonable in that lesson item difficulty is the actual 
averaged probability of selecting that alternative as the correct response during the lesson for that population of 
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learners. To our knowledge, this is the first investigation to utilize lesson item difficulty values as a measure of 
input and output activation levels. 

In the delta rule equation, feedback impacts learning in the term (1* - a^t). Customarily, the values for to 
and ao Ul are constrained between 0 and 1 (McLeod, Plunkett, & Rolls, 1998). The value for t 0 equals 1 if the 
activation level of the output unit matches the desired response (i.e., with correct responses) and t 0 equals zero if the 
activation level of the output unit does not match the desired response (i.e., with incorrect responses). So with 
correct responses the association weight increases since (1 - a^) is positive , while with incorrect responses the 
association weight decreases since (0 - a^,) is negative . 

In other words, when feedback is provided as part of the responding' instance, correct responses are 
strengthened and incorrect responses weakened. The amount of increment or decrement can be determined by the 
delta rule. Thus, given lesson item difficulties (initial a^), the delta rule should be able to predict posttest item 
difficulties (a^ after feedback). What values would the delta rule provide for ILRs for immediate and for delayed 
feedback? In the present investigation, first the delta rule would predict that for correct lesson responses, memory of 
ILRs and of correct responses would be strengthened in general for both immediate and delayed feedback, since to = 
1 and so (t 0 - a out ) is positive. Second, for lesson errors, the ILR association with the item stem would be weakened 
for immediate feedback since ^ = 0 and (^ - a out ) is negative, but not for delayed feedback. 

For delayed feedback, the connectionist model would predict that ILR errors would actually be 
strengthened. In associative learning in living systems, there is a small window of time while the specific input 
pattern is activated lasting probably less than 4 seconds (Shanks, Pearson, & Dickerson, 1989) when those 
associations can be strengthened or weakened. Immediate feedback provides the necessary teacher feedback 
information within this time frame while delayed feedback does not. Specifically, with delayed feedback, since 
corrective feedback is not immediately provided, the learning rule association process would act as though the error 
response is correct, thus strengthening the association weight of the error (see Figure 1). 




0.20 0.40 0.60 0.80 



Lesson Item Difficulty (p) 

Figure 1. Predicted retention test values generated by the delta rule across a range of lesson item difficulty values 
(from Clariana, 1999). Predicted retention test memory of Initial Lesson Responses (ILR) are shown as dashed 
lines and Correct Responses (CR) are shown as solid lines for delayed feedback (DF) and immediate feedback (IF). 
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Based on the connectionist model of feedback timing described by Clariana (1999), several hypotheses can 
be stated: (1) Retention test memory of ILRs will be considerably greater for delayed feedback than for immediate 
feedback at all item difficulty levels (see dashed lines in Figure 1). (2) Both types of feedback will obtain the 
greatest lesson to posttest gain with difficult lesson items. (3) Retention test memory of correct responses will vary 
across the range of possible lesson item difficulty values for the delayed and immediate feedback forms, with 
immediate feedback slightly better than delayed feedback with more difficult lesson items and delayed feedback 
slightly better than immediate feedback with easier lesson items (see the solid lines in Figure 1). 

In addition, it may be possible to separate any observed effects of immediacy versus multiple-exposures by 
including a type of feedback, multiple-try immediate feedback (MTF), which has aspects of both (Bangert-Drowns, 
Kulik, Kulik, & Morgan, 1991; Clariana, 1993). MTF is like the more common form of immediate feedback 
(single-try feedback, STF) in terms of immediacy of feedback timing. But in terms of multiple explicit stimulus 
exposures, multiple-try feedback is like delayed feedback (DF), at least with lesson errors. 

Method 

The available sample for this study included students from three high school social studies classes (n= 87) 
from a small town in a northeastern state. The students were mostly sophomores, with a few juniors. A number of 
students chose not to participate, some were absent, a few forgot to return their signed consent forms, and three 
students' data were dropped due to incomplete data, yielding a final sample of 52 students. The final sample 
contained more females (71%) than males. 

The computer-based lesson material consisted of eight reading passages and 36 five-alternative multiple- 
choice questions from the Nelson-Denny Reading Test, Form E (Brown, Bennett, & Hanna, 1981; with permission of 
the publisher). These text passages and questions were chosen based on the quality of the text and questions, their 
high reliability, the availability of extensive test reliability metrics, and content appropriateness for this audience. 
Field trials by the developers involved more than 14,000 students (Brown, Bennett, & Hanna, 1981). Currently, 
these materials are extensively used in the field and are readily available from the publishers. 

The 36 questions consisted of 18 verbatim questions and 18 inferential questions, similar to Peeck and 
Tillema (1978). Verbatim questions relate directly to one proposition in the instructional text. For example, the text 
may say, "Daytimes Robert Browning walked with Elizabeth's little dog Flush, but he seldom could be lured from 
his home evenings", with the associated verbatim question, "Flush was the name of the Brownings': A. cat B. dog 
C. canary D. gold fish E. thrush?". Inferential questions relate to multiple propositions in the text and can be 
answered by considering the passage as a whole. For example, the inferential question, "The authors of this passage 
placed most attention on the Brownings: A. literary efforts B. personal relationship C. social contacts D. 
problems E. early meeting?", can be answered by considering the entire passage. Thus, inferential questions have 
many indirect connections to the instructional text, while verbatim questions have one or only a few direct 
connections to the text. 

The computer-based lessons were developed in HyperCard and delivered on Macintosh computers. The 
three alternative computer-based lessons were identical except for the type of feedback that was presented after the 
learner responded to multiple-choice questions. For all treatments, each of the eight text passages was presented in 
turn in scrolling text windows. After reading a text passage, the learner would respond to either four or eight five- 
alternative multiple-choice questions (the longer passages had more questions). The learner would then proceed to 
the next text passage reading the text and answering the questions at his or her own pace. Simple navigation buttons 
along the bottom of every screen allowed the learner to easily move back and forth between text and questions at 
anytime. 

The three alternate treatments were delayed feedback (DF), single-try immediate feedback (STF), and 
multiple-try immediate feedback (MTF). Note that since retention test memory of initial lesson responses is a 
dependent variable in this study, in order to prevent rehearsal of initial lesson responses, the final feedback screen 
for each item in all three treatments did not include item distractors (Sassenrath & Yonge, 1969; Sturges, 1969). 

The STF treatment provided the correct response immediately after one learner response, whether the 
response was right or wrong. After a response, an arrow would point to the correct alternative and the learner was 
told "Right" when correct and "No, here is the answer" when wrong. In either case, the stem and correct answer 
were displayed, the item distractors were not shown. The MTF treatment provided the correct response immediately 
after a correct response like STF. However, with an incorrect response, the learner was told "No, try again" and 
continued to select answers until the correct response was selected, then an arrow would point to the correct 
response and the learner was told "Right". Then the stem and correct answer were displayed, the item distractors 
were not shown. Note that STF and MTF are identical when the learner's initial response is correct, but obviously 
differ when the learner's response is incorrect. The DF treatment required the learner to respond to questions and 
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move on without any immediate feedback. After all text passage and questions were completed, then all of the items 
were presented again in the original order. Only the correct responses were shown with each question, the 
distractors were not shown, and the student could only read the display. Thus the DF feedback screen displays were 
identical to the final feedback screen displays provided for STF and MTF, only the DF feedback screen displays 
were presented after the entire lesson rather than immediately with each item. 

The purpose and requirements of the study were explained in three classes of students all taught by the 
same teacher. Those students choosing to participate collected consent forms to be signed by a parent or guardian. 
About a week later, participants moved to the school computer room during their social studies class, and were 
randomly assigned to one of the three computer-based treatments, STF, MTF, or DF. One day later, participants 
completed the paper-and-pencil 24-hour retention test in class. 

The retention test given a day after the lesson was designed to measure memory of initial lesson responses 
and of the correct responses. This paper-based retention test used the same 36 multiple-choice items that were used 
in the computer-based lesson. These 36-items fell into two groups of 18 items each, verbatim and inferential. These 
two groups were further blocked into three categories by lesson item difficulty, easy (M = -87), mid-range (M = -72), 
and difficult (M = .50), with each block containing six items. The associated reading passages were not made 
available to the students during retention testing. There were four or five questions on each 8.5- by 1 1-inch page. 
Each question had two blanks, one blank labeled “1st” for the initial lesson response and one blank labeled “C” for 
the correct response. 

The retention test contained the following instructions, “Note that each question has two blanks. The first 
blank is a check to see if you can remember the first answer that you gave during the computer lesson. The second 
blank is for the correct response to the question.” These instructions were read aloud by the teacher. The teacher 
answered questions about how to complete the test, and then students were given as much time as needed to finish. 

The total amount of time spent completing the lesson was also of interest. Lesson time data for each 
student was collected by the computer program, and included total time from the display of the first screen until the 
student exited the lesson. 

Results 

Dependent variables were lesson time, and retention test recognition memory of initial lesson responses and 
of correct responses with two kinds of questions (verbatim and inferential) across three levels of lesson item 
difficulty (difficult, mid-range, and easy). These data were analyzed by separate analysis of variance, and 
probabilities were evaluated more conservatively using Greenhouse-Geisser and Huynh-Feldt corrections 
automatically provided by the SYSTAT 8.0 (1998) analysis package. Lesson time data and retention test means and 
standard deviations for each treatment group at each item kind and difficulty level are provided in Table 1 . 

Lesson scores were analyzed as a check of initial group equivalence. The random assignment was judged 
successful with overall lesson scores of 24.4 (of 36 maximum) for the STF group, 25.4 for the MTF group, and 25.5 
for the DF group. The comparison of these means using analysis of variance was non-significant, F(l, 51) = 0.43, p 
= .65. 

Retention test data were analyzed by a mixed 3 x (2 x 2 x 3) analysis of variance with one between-subjects 
factor, feedback condition (DF, MTF, or STF), and three within-subjects factors, type of response (retention of 
initial lesson response and of the correct response), kind of question (verbatim and inferential), and lesson item 
difficulty block (difficult, mid-values, and easy). The interaction of feedback and type of response was significant, 
F (2, 49) = 7.15, MSE = 0.023, p < .01. Follow-up Scheffe' test showed that the DF treatment group mean for ILR 
(M = 0.84) was significantly larger than the STF group mean for ILR (M = 0.73), no other mean comparisons were 
significant. 
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Table 1 

Lesson time data (in seconds), and lesson and retention test means and standard deviations (in parentheses) for 
each treatment group for each item kind and difficulty level. 









Lesson 




Retention Test 




Treatment 


Lesson time 


Item difficulty 


V 


I 


ILR-V 


ILR -I 


CR-V 


CR-I 


STF (n=l 8) 


1577 


Difficult 


0.46 


0.44 


0.66 


0.66 


0.67 


0.83 


(280.6) 




(0.20) 


(0.10) 


(0.18) 


(0.26) 


(0.22) 


(0.18) 






Mid-range 


0.85 


0.57 


0.76 


0.67 


0.79 


0.67 








(0.07) 


(0.13) 


(0.18) 


(0.19) 


(0.18) 


(0.21) 






Easy 


0.92 


0.82 


0.87 


0.78 


0.91 


0.84 








(0.08) 


(0.11) 


(0.12) 


(0.18) 


(0.09) 


(0.12) 


MTF (n=17) 


1554 


Difficult 


0.51 


0.55 


0.71 


0.67 


0.71 


0.72 


(332.5) 




(0.20) 


(0.16) 


(0.24) 


(0.17) 


(0.20) 


(0.21) 






Mid-range 


0.77 


0.65 


0.79 


0.72 


0.81 


0.74 






(0.07) 


(0.04) 


(0.18) 


(0.16) 


(0.17) 


(0.17) 






Easy 


0.91 


0.83 


0.86 


0.81 


0.91 


0.81 








(0.09) 


(0.09) 


(0.12) 


(0.18) 


(0.12) 


(0.18) 


DF(n=17) 


1899 


Difficult 


0.48 


0.55 


0.80 


0.81 


0.68 


0.68 


(409.9) 




(0.18) 


(0.14) 


(0.18) 


(0.22) 


(0.25) 


(0.27) 






Mid-range 


0.85 


0.65 


0.84 


0.78 


0.85 


0.78 






(0.06) 


(0.07) 


(0.15) 


(0.18) 


(0.14) 


(0.21) 






Easy 


0.94 


0.78 


0.93 


0.87 


0.89 


0.82 






(0.05) 


(0.08) 


(0.09) 


(0.16) 


(0.10) 


(0.17) 



Note, Lesson time is in seconds; standard deviations in parenthesis; V = Verbatim questions, I - Inferential 
questions; ILR = remembers initial lesson response; CR = recognize correct response; STF = Single-try feedback, 
MTF = Multiple-try feedback, DF = Delayed feedback. Each item difficulty range is the average of 6 items. 



A significant effect was obtained for kind of question, F(l, 49) = 8.29, MSE = 0.035, p < .01, indicating 
that the verbatim retention test mean (M = 0.80) was greater than the inferential retention test mean (M = 0.76), 
which simply reflects the lesson values for verbatim (M = 0.74) and inferential (M = 0.65) questions. Next, a 
significant effect was obtained for question difficulty, F(2, 98) = 32.20, MSE = 0.035, p < .01. The retention test 
means for each item difficulty level are: easy items (M = 0.86), mid-range difficulty items (M = 0.77), and difficult 
items (M = 0.71). As with kind of question above, these retention test values simply reflect lesson values which are: 
easy items (M = 0.87), mid-range difficulty items (M = 0.72), and difficult items (M = 0.50). Though these findings 
are significant, they have little practical meaning. 

The three-way interaction of feedback, type of response, and question difficulty was significant, F(4, 98) = 
3.52, MSE = 0.015, p < .01. To further examine this complex three-way interaction, two separate follow-up 
ANOVAs of retention test data are reported below, one of initial lesson response data and one of correct response 
data. But first this three-way interaction is graphically compared to the delta rule predicted values (see Figure 2) to 
set the stage for the follow-up analyses. 

Since the observed retention data shown in Figure 2 consists of multiple-choice questions, these data were 
corrected for guessing in order to be consistent with the predicted values. The correction for guessing formula from 
Nitko (1996) is corrected score = R - W/(n-l) where R is raw score, W is number wrong, and n is the number of 
multiple-choice alternatives. The lines showing the predicted and observed retention for initial lesson responses 
(left panel of Figure 2) are highly similar in both magnitude and form, suggesting that the connectionist model 
presented by Clariana (1999) may adequately account for 24-hour retention of initial learner responses. The lines 
showing the predicted and observed retention of correct responses (see right panel of Figure 2) are generally similar 
in shape but are not similar in magnitude. Specifically, the predicted correct response values over-estimate the 
observed values. 
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Figure 2. Lesson Response and Correct Response predicted retention test values (dashed lines) for immediate 
feedback (IF) and delayed feedback (DF) compared to observed data corrected for guessing (solid lines) for delayed 
feedback (DF), multiple-try feedback (MTF), and single-try immediate feedback (STF) 

Follow-up Analysis of Retention Test ILR Data 

Retention test ILR data were analyzed by a mixed 3 x (2 x 3) analysis of variance with one between- 
subjects factor, feedback condition (DF, MTF, or STF), and two within-subjects factors, kind of question (verbatim 
and inferential) and lesson item difficulty (most difficult, mid-range, and easy). A significant effect was obtained 
for feedback, F(2, 49) = 4.13, MSE = 0.08 1 , p < .05, as already reported above that retention of ILRs was greater for 
DF compared to STF (see Figure 2), in addition MTF was more like STF than like DF. 

A significant effect was also observed for kind of question F(2, 49) = 8.24, MSE = 0.024, p < .01 and for 
item difficulty blocks F(2, 98) = 20.24, MSE = 0.025, p < .01. Retention test memory of ILRs for verbatim 
questions (M = 0.80) was greater than memory of inferential questions (M - 0.75). Retention test memory of ILRs 
for difficult items (M = 0.72) and for mid-difficulty items (M “ 0.76) was less than memory of easy items (M “ 
0.85). As above, though these two findings are significant, they have little practical meaning. 

Follow-up Analysis of Retention Test Correct Response Data 

Retention test correct response data were analyzed by the same mixed 3 x (2 x 3) analysis of variance. 
Three findings reached significance. A significant effect was obtained for item difficulty blocks, F(2, 98) = 24.74, 
MSE = 0.025, p < .01. Scheffe' tests show that retention test memory of correct responses for difficult items (M = 
0.71) and for mid-range items (M = 0.77) were both less than memory of easy items (M = 0.87), a finding of little 
practical interest. 

More importantly, the interaction of feedback and item difficulty was significant, F(4, 98) = 2.54, MSE = 
0.025, p < .05 (Greenhouse-Geisser p = 0.05; Huynh-Feldt p = 0.04). Although this disordinal interaction was 
directionally consistent with the delta rule predictions, with STF best for difficult items and DF best for easy items 
(see right panel of Figure 2), follow-up Scheffe' test obtained no significant differences for type of feedback within 
each question level. Thus the hypothesized differences between DF and STF at different item difficulty levels were 
too small to be considered reliable. 

The interaction between kind of question and lesson item difficulty was significant, F(2, 98) = 7.62, MSE = 
0.023, p < .01. Though inferential lesson questions appear to be slightly more effective than verbatim questions for 
difficult lesson items, follow-up Scheffe' test obtained no significant differences for type of feedback within each 
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question level. Thus a possible instructional advantage of inferential questions over verbatim questions (per Merrill, 
1987) is too small to be considered reliable. 

Discussion 

The first hypothesis was confirmed that retention of initial lesson responses is greater for delayed feedback 
compared to immediate feedback across all item difficulties, but especially with difficult lesson items. The 
association weights of ILR errors increased (see dashed line in Figure 2). Further, the observed retention of initial 
lesson responses for STF and DF were very similar to their corresponding delta rule predicted values (see left panel 
of Figure 2). This finding provides empirical support for the potential of a connectionist model to predict 
instructional feedback effects. 

The practical value of hypothesis one for instructional design is that in some learning situations, it is critical 
to remember initial lesson responses, especially if answers are not absolutely "right" or "wrong" but serve as 
learning transitions to broader understanding. For example, in discovery learning situations, learners are required to 
remember and use previous responses. In such cases, delayed feedback or even no feedback would allow learners to 
maintain initial lesson responses at a greater rate than with immediate feedback. On the other hand, immediate 
feedback involves a trade-off between increasing correct response associations at the expense of forgetting other 
responses, and these other responses are likely more meaningful to the learner even though "incorrect". In the many 
situations where it is critical to strengthen the correct response and diminish the incorrect response, then immediate 
feedback would be better. 

The second hypothesis that feedback has its greatest effect with difficult lesson items was confirmed. 
Lesson to retention test change at each lesson item difficulty block expressed in effect sizes (ES), calculated as the 
difference between lesson and retention score divided by the standard deviation of the lesson score, are: for easy 
lesson items, ES = -.06, for mid-range lesson items, ES = .35, and for difficult lesson items, ES = 1.17. Previous 
studies have provided the groundwork for this finding by showing that feedback has its greatest effect with difficult 
items (Sturges, 1978). For example, Bangert-Drowns, Kulik, Kulik, and Morgan (1991) state, "If feedback’s primary 
importance is the correction of errors, then one would expect to see larger effects for instruction with higher error 
rates. This is exactly what happens" (p.230). Thus, future feedback investigations must consider and control lesson 
item difficulty, or else results may be confounded by lesson difficulty. 

The third hypothesis, that feedback timing interacts with lesson item difficulty, was not supported. 
However the mean differences were in the right direction with STF best with difficult items and DF best with easier 
items (see the right panel of Figure 2). This result could be anticipated in that lesson items were not difficult enough 
to produce the interaction, the average lesson item difficulty for the most difficult question block in this 
investigation was M = 0.50. Note that the delta rule predicted difference between immediate and delayed feedback 
would be most pronounced for lesson item difficulties less than about 0.40 (see Figure 2), and are actually predicted 
to be identical for lesson item difficulties near 0.50. Thus, to adequately test this hypothesis, an unusually difficult 
lesson would be required. Pragmatically, for computer-based lessons that use multiple-choice questions of 
reasonable difficulty, immediate and delayed feedback groups should obtain similar posttest scores, with perhaps a 
slight advantage for delayed feedback. 

Though the delta rule predicted ILR values well, this approach overestimated posttest correct response 
values. Using trend analysis, the values a aj n = 0.5 and t 0 = 0.85 will adjust the delta rule predictions for posttest 
correct responses to more closely match the observed data. Future investigations should consider whether these 
variables of the delta rule are more constant for classes of learner response, or are more variable, perhaps related to 
individual learner variables. 

What are the effects of feedback immediacy and of multiple exposures? MTF was much more like STF in 
retention test memory of ILRs indicating that feedback immediacy acts to reduce memory of ILR errors, a 
retroactive interference effect. But MTF rather than clearly mirroring STF (immediacy) or DF (multiple item 
exposure) generally fell midway between STF and MTF, indicating that both feedback timing and number of 
exposures combine or interact to impact retention test memory, especially for memory of correct responses. This 
combination or interaction of immediacy and multiple-exposures is of theoretical interest, and so should be 
addressed by additional experimentation. 

The findings of this study involve only retention test recognition learning outcomes and should not be 
generalized to other types of learning outcomes, such as recall. Additional research should consider the application 
of a connectionist model for predicting the effects of feedback on higher-level learning outcomes. 

As a footnote, Kulhavy and Stock (1989) have described an information processing explanation of 
feedback effects based on servocontrol theory (which describes the interaction between system output, sensors, 
feedback from the sensors, and mechanical devices that impact output). Their model views learner's response 
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confidence as a metacognitive component that controls or at least strongly influences how feedback information is 
processed by the learner. So far (since Sturges, 1978), response confidence studies have shown mixed results 
(Bangert-Drowns, Kulik, Kulik, & Morgan, 1991; Mory, 1992; 1994), so whether response confidence serves a 
metacognitive function in feedback processing is unknown. Requiring a learner to consciously consider the 
confidence of every lesson response would obviously alter the expected and normal pattern of a lesson, for example 
increasing the amount of time the learner takes with each item. Further, asking the learner for response confidence 
at each lesson response may be distracting, and could disrupt the learning process. Also, it has been suggested by 
Mory (1994) that learners' self-report of response confidence is inaccurate in some cases (see also Metcalfe, 1986). 
For these reasons, response confidence was not used in this present investigation. However, response confidence 
could serve as a logical alternative measure of initial lesson output activation level, a^, in the delta rule calculations. 
Thus, future investigation of the possible metacognitive effects of response confidence may obtain added insight by 
applying a connectionist model. 

(Note : A more detailed version of this manuscript has been accepted for publication in ETR&D. Special thanks to 
Dave Jonassen for his assistance in forming the idea for the study and for directing several doctoral students my 
way, and also thanks to Steve Ross and the ETR&D reviewers for their suggestions and recommendation for the 
manuscript.) 



References 

Ausubel, D. P. (1968). Educational psychology: A cognitive view . New York: Holt, Rinehart and Winston. 
Bangert-Drowns, R. L., Kulik, C. C., Kulik, J. A., & Morgan, M. (1991). The instructional effect of 
feedback in test-like events. Review of Educational Research, 61 (2), 213-238. 

Brown, J . I., Bennett, J. M., & Hanna, G. (1981). The Nelson-Penny Reading Test: Examiner's manual . 
Itasca, IL: Riverside Publishing. 

Bruner, J. (1990). Acts of meaning . Cambridge, MA: Harvard University Press. 

Clariana, R. B. (1999). Differential memory effects for immediate and delayed feedback: A delta rule 
explanation of feedback timing effects . A Presentation at the Annual Convention of the Association for Educational 
Communications and Technology, February, 1999 in Houston, TX. (ERIC Document Reproduction Center ED 430 
550). 

Clariana, R. B. (1993). A review of multiple-try feedback in traditional and computer-based instruction. 
Journal of Computer-Based Instruction , 20 , 67-74. 

Crocker, L. M. & Algina, J. (1986). Introduction to classical and modem test theory . New York: Holt, 
Rinehart and Winston. 

Elman, J. L. (1993). Learning and development in neural networks: The importance of starting small. 
Cognition , 48, 7 1 -99. 

Haberlandt, K. (1997). Cognitive Psychology 2 nd edition . Boston, MA: Allyn and Bacon. 

Kulhavy, R. W., & Stock, W. A. (1989). Feedback in written instruction: The place of response certitude. 
Educational Psychology Review. 1 (4), 279-308. 

McLeod, P., Plunkett, K., & Rolls, E. T. (1998). Introduction to connectionist modeling of cognitive 
processes . Oxford, UK: Oxford University Press. 

Merrill, J. (1987). Levels of questioning and forms of feedback: Instructional factors in courseware design. 
Journal of Computer-Based Instruction , 14 , 18-22. 

Metcalfe, J. (1986). Feeling of knowing in memory and problem solving. Journal of Experimental 
Psychology: Learning. Memory and Cognition . 12 , 288-294. 

Mory, E. H. (1992). The use of informational feedback in instruction: Implications for future research. 
Educational Technology Research and Development, 40 (3), 5-20. 

Mory, E. H. (1994). Adaptive feedback in computer-based instruction: Effects of response certitude on 
performance, feedback-study time, and efficiency. Journal of Educational Computing Research, 1 1 (3), 263-290. 

Nitko, A. J. (1996). Educational assessment of students, second edition . Englewood Cliffs, NJ: Prentice- 

Hall. 

Peeck, J., & Tillema, H. H. (1978). Delay of feedback and retention of correct and incorrect responses. 
Journal of Experimental Education, 38 . 171-178. 

Plunkett K., & Marchman, V. A. (1993). From rote learning to system building: Acquiring verb morphology 
in children and connectionist nets. Cognition , 48, 21-69. 




90 

9 



Plunkett K., & Marchman, V. A. (1996). Learning from a connectionist model of the acquisition of the 
English past tense. Cognition . 61, 299-308. 

Sassenrath, J. M,, & Yonge, G. D. (1969). Effects of delayed information feedback and feedback cues in 
learning on delayed retention. Journal of Educational PsychologY,_50 , 174-177. 

Seidenberg, M., & McClelland, J. (1989). A distributed model of word recognition and naming. 
Psychological Review . 96. 523-568. 

Shanks, D. R. (1995). The psychology of associative learning . Cambridge, UK: Cambridge University 

Press. 

Shanks, D. R., Pearson, S. M., & Dickinson, A. (1989).Temporal contiguity and the judgment of causality. 
Quarterly Journal of Experimental Psychology . 4 IB , 139-159. 

Sturges, P. T. (1969). Verbal retention as a function of the informativeness and delay of informative 
feedback. Journal of Educational Psychology. 60 (1), 11-14. 

Sturges, P. T. (1978). Delay of informative feedback in computer-assisted testing. Journal of Educational 
Psychology. 70 (3), 378-387. 

SYSTAT 8.0 [Computer software]. (1998). Evanston, IL: SPSS Inc. 

Widrow, B., & Hoff, M. E. (1960). Adaptive switching circuits. 1 960 IRE WESCON Convention Record 
(Pt. 4\ 96-104. 



o 

ERIC 



91 10 



U.S. Department of Education 

Office of Educational Research and Improvement (OERI) 
National Library of Education (NLE) 
Educational Resources Information Center (ERIC) 




NOTICE 

REPRODUCTION BASTS 



This document is covered by a signed “Reproduction Release 
(Blanket) form (on file within the ERIC system), encompassing all 
or classes of documents from its source organization and, therefore, 
does not require a “Specific Document” Release form. 



This document is Federally-funded, or carries its own permission to 
reproduce, or is otherwise in the public domain and, therefore, may 
be reproduced by ERIC without a signed Reproduction Release form 
(either “Specific Document” or “Blanket”). 




